1. Technical Field
The present invention relates to an image processing technique and specifically to a technique for realizing image conversion, such as upsizing, downsizing, and the like, image compression, and texture mapping.
2. Background Art
Due to digitalization of image-related devices and networks, connection of any image-related devices has been possible, and the flexibility of image exchange has been improving. Further, an environment for users to freely handle images (pictures) without being restricted by difference in system has been constructed. For example, users can print images captured by a digital still camera, lay open the images on a network, and view the images on a household television set.
In the meanwhile, systems need to comply with various image formats and, naturally, a higher technique is required in image format conversions. For example, conversion of image size frequently occurs and, in such a case, an up-converter (converter for increasing the number of pixels or the number of lines) and down-converter (converter for decreasing the number of pixels or the number of lines) are necessary. For example, in the case of printing an image with the resolution of 600 dpi on an A4 (297 mm×210 mm) paper, a document of 7128 pixels×5040 lines is necessary, but many digital still cameras are short of this size, and therefore, an up-converter is necessary. On the other hand, an image laid open on a network does not have a definite final output form. Therefore, every time an output device is selected, the image needs to be converted to have an image size compliant with the selected output device. As for the household television set, since digital terrestrial broadcasting services have been started, conventional standard television images and HD (High Definition) television images are mixed. Thus, conversion of image size is frequently carried out.
If there are various image sizes, the significance of scalability in image compression becomes higher. The scalability means, in some cases, extracting standard television image data and, in other cases, extracting HD television image data from one bit stream. In other words, the scalability means the flexibility in extracting image data of various image sizes. If the scalability is secured, it is not necessary to prepare a transmission path for every format, and only a small transmission capacity is required.
Image conversion, such as image upsizing, image downsizing, and the like, is frequently employed for texture mapping in computer graphics (designs and patterns appearing on a photograph subject are generically referred to as “texture(s)”). Texture mapping is a method for expressing the patterns and textures on the surface of an object by placing two-dimensional images over the surface of a three-dimensional object created in a computer. To place the two-dimensional images so as to comply with the direction of the surface of the three-dimensional object, it is necessary to carry out processes, such as upsizing, downsizing, deformation, rotation, etc., on the two-dimensional images (see non-patent document 1).
Conventionally, the processes, such as image upsizing, image downsizing, image compression, etc., take advantage of the difference in brightness among a plurality of pixels.
In image upsizing, brightness values are interpolated according to a bilinear method, bicubic method, or the like, in order to newly generate image data which does not exist at the time of sampling (see non-patent document 1). In interpolation, only intermediate values of sampling data can be generated, and therefore, the sharpness of an edge, or the like, shows a tendency to deteriorate. In view of such, there has been a technique wherein an interpolated image is used as an initial upsized image and, thereafter, an edge portion is extracted to emphasize only the edge (disclosed in non-patent documents 2 and 3). Especially in non-patent document 3, multi-resolution representation and Lipschitz index are employed, such that an edge is selectively emphasized according to the sharpness of the edge.
In image downsizing, some pixels are deleted. If pixels which are at separate positions before downsizing are placed adjacent to each other, continuity is marred so that moire fringes occur. To avoid such a problem, in general, the data is subjected to a low-pass filter before some pixels are deleted to obtain a smooth brightness variation, and thereafter the some pixels are deleted.
In image compression, a high correlation in brightness between adjacent pixels is utilized. To express the correlation in brightness, a spatial frequency component is divided into quadrature components. In general, discrete cosine transform is utilized for orthogonal transformation, and energy is concentrated at low-frequency terms because of high correlation in brightness between adjacent pixels. Thus, high-frequency terms are deleted, whereby image information is compressed (see non-patent document 4).    [Patent Document 1] Japanese Laid-Open Patent Publication No. 2005-149390.    [Non-patent Document 1] Shinji Araya, Clear commentary on 3D computer graphics, Kyoritsu Shuppan Co., Ltd., pp. 144-145, Sep. 25, 2003.    [Non-patent Document 2] H. Greenspan, C. H. Anderson, “Image enhancement by non-linear extrapolation in frequency space”, SPIE Vol. 2182 Image and Video Processing II, 1994.    [Non-patent Document 3] Makoto Nakashizu, et al., “Increase in resolution of image in multi-scale brightness gradient plane”, The IEICE transactions (Japanese Edition), D-II Vol. J81-D-II No. 10 pp. 2249-2258, October 1998.    [Non-patent Document 4] Broadband+Mobile Standard MPEG textbook: Point-by-point illustration edited by Multimedia Communication Study Group, ASCII Corporation, pp. 25-29, Feb. 11, 2003.    [Non-patent Document 5] Image Processing Handbook edited by Image Processing Handbook Editorial Committee, Shokodo Co., Ltd., pp. 393, Jun. 8, 1987.    [Non-patent Document 6] Shinji Umeyama, “Separation of diffuse and specular components of surface reflection—using multiple observations through a polarizer and probabilistic independence property”, Symposium on image recognition and understanding 2002, pp. I-469-pp. I-476, 2002.